Outline of extreme test sensitivity analysis

Extreme conditions test is a stress test where input parameters are varied beyond probabilistic ranges, and model-generated outcome behaviour is examined for consistency with the real or anticipated behaviour of the system (). Models that operate with logical consistency outside of “normal” conditions have greater consistency, building confidence in the model output.

Benefits of extreme conditions test:

1. It can be used to discover flaws in the model’s conceptual logic.
2. Aids in identifying non-linear and asymptotic behaviours that can be used to validate the model behaviour and highlight possible impactful policy levers.
3. enhances the model’s usefulness by exploring model behaviour of systems that operate outside of contextual-historical observations, i.e. populations with differing characteristics.

Outcome Measure

The plot below describes how Elementary Effect (EE) are used to summarise sensitivity analysis results.

Panel A shows an example of a model-generated outcome; over modelled time, the input assumptions (\(x\)) are varied between \(\pm\) 20% of it’s initial value. The model results at the end of the model are used to calculate the EE (panel B).

The EE is calculated for each model output result to create a sample of EE. The mean and standard deviation are used to summarise the change in outcome per percentage point shift in input assumption.

\[ EE_{i} = \frac{f(x-\Delta_{i})- f(x)}{\Delta_{i}}\]

\[ \mu_{EE}= \frac{1}{n} \sum_{i=1}^{n} EE_{i}, \hspace{1cm} \sigma_{EE}=\frac{1}{(n-1)} \sum_{i=1}^{n} (EE_{i} - \mu_{EE})^2\]

Example plot for analysis outcome

How to interpret the results

A summary plot (panel A) shows the summary relationships between variation in the input variables (overweight males body weight aged between 6 and 8) and the highlighted outcome, the prevalence of obesity.

The individual input-output relationships are plotted in the facets of panel B. Each sub-plot shows the model-generated prevalence of obesity at the end of the model for each input assumption.

As the assumed body weight for overweight males aged between 6 and 8 is varied, the plots in panel B show no impact on the prevalence of obesity for 2 and 2 to 5-year-olds, which is also reflected in the zero association in the summary plot. The prevalence of obesity for 6 to 8-year-olds and above have a negative association with 6 to 8-year-old overweight males assumed body weight. The largest associations were in the 9 to 11-year-olds, and the strength of the association tappers out for older age groups.

For every percentage point change in the assumed body weight for overweight males aged 6 to 8 years old, there is, on average, a 0.25% reduction in the prevalence of obesity in 6 to 8-year-olds. This suggests that 25% of the input variability is observed in the outcome.

Overall Ranking of sensitivity

The 649 input assumptions tested can be summarised into 70 variables by taking the mean of age-gender-BMI specific inputs, resulting in 70 overarching variables. These are ranked in the following table.

Body Weight

Change in Male Body Weight input assumptions

Changes in base model assumptions for body weight alter how changes in energy imbalance impact BMI category prevalence estimates, making the model insensitive to changes based on assumed energy surplus. The figure below shows the elementary effect for each local sensitivity analysis changing the assumed male body weight.

• Changes in the body weight assumptions reduced the flow of individuals to higher BMI categories, creating higher healthy weight prevalence.
• Younger ages impact the BMI outcomes for all older age groups.
• Older age groups do not impact younger outcomes.
• Changes in each BMI category mainly impact the outcome of the BMI category being changed and the neighbouring BMI category.
  • Change in the assumptions for Healthy weight impacts the underweight and healthy weight and overweight outcome.
  • Change in the assumptions for overweight mainly impacts the outcome for all BMI categories.
  • Changing the obesity assumptions change the obesity and overweight outcomes.

Change in Female Body Weight input assumptions

There are similar relationships, as noted in the male assumptions.

Additionally;

• The results show the intergenerational effects.
  • Changes in the female assumptions slightly impact the younger age group.
• The intergenerational effect is stronger when varying assumptions in age groups with higher fertility rates
  • For each PP increase in body weight for Underweight & Healthy weights 20 to 24 resulted in a
    • 0.57% increase in the Underweight & Healthy 20 to 24 years olds
    • 0.09% increase in Underweight & Healthy 2-year-old males (15.9% of the adults’ effect)
  • For each PP increase in body weight for Underweight & Healthy weight 40 to 44 year-olds resulted in a
    • 0.39% increase in the Underweight & Healthy 40 to 44-year-olds
    • 0.003% increase in Underweight & Healthy 2-year-old males (0.87% of the adults’ effect)

This suggests that any prevention intervention for early childhood should be targeted at age groups with higher fertility rates to have a larger impact.

Height

Similar to changes in the assumed body weight for each cohort, changes in height impact how influential energy surplus and deficits result in changes in the flows between BMI categories. However, height is the least influential in the model compared to body weight.

Change in Male height input assumptions

Change in Female height input assumptions

Growth Function kJ/day

Changes in the assumption Kj/day needed for growth resulting in relatively small changes in the BMI outcomes. The largest impact occurred in adolescent age groups where the growth assumptions where the largest. The larger observed impact in males was in 9-11 year olds and 12-15 year olds, where on average 1 Kj resulted in 0.2% change in the outcome.

Macronutrient energy density

Each food group is broken down into macronutrients; carbohydrates, protein, fats and sugars. The energy from a gram of macronutrients is an assumed model input. This assumption impacts each food group for each age-gender-BMI group.

• The input of macronutrients becomes more sensitive for the older age group. This is due to the cumulative impact of the population’s life course.
• Higher kJ/g leads to higher daily total dietary intake leading to a higher prevalence of overweight and obesity.
• For every percentage point change in the assumed energy density for dietary fats, there was, on average, a 1.08% increase in the prevalence of obesity in 12 14-year-olds for females and a 1.15% increase for males.
• This effect was 1.98% and 2.72% for 40 44 year olds females and males, respectively.

Explanation of summary results

Thermic effect of food (TEF)

The thermic effect of food (TEF) is the proportion of the energy used for digestion. A higher TEF means less dietary energy is available after digestion. TEF input assumptions impact the whole population, translating to a highly sensitivity input assumption.

• Change in TEF, cumulative over the life course leading to a higher impact for older age groups.
• 1 percentage point (0.0015 TEF units) increase in the TEF of carbohydrate results in a -0.06 to -0.52% change in obesity.
  • -0.44 to -3.51% change in obesity per 0.01 TEF.
• 1 percentage point (0.0015 TEF units) increase in the TEF of Fat results in a -0.07 to -0.59% change in obesity.
  • -0.48 to -3.99% change in obesity per 0.01 TEF.
• 1 percentage point (0.002 TEF units) increase in the TEF of Protein results in a -0.06 to -0.49% change in obesity.
  • -0.30 to -2.45% change in obesity per 0.01 TEF.
• 1 percentage point (0.0007 TEF units) increase in the TEF of Sugar results in a -0.03 to -0.27% change in obesity.
  • -0.43 to -3.90% change in obesity per 0.01 TEF.

Explanation of summary results

Adult to Adult Social Transmission

Adult to Adult social transmission assumptions impact how changes in physical activity (PAL) and dietary behaviours are transmitted to other adults through role modelling. This assumption has little impact on the output. However, because this variable is dependent on change of behaviour, role-modelling likely interactions with intervention effects.

Adult to Child Social Transmission

Similar to “Adult to Adult” role-modelling, BMI outcome has little impact when Adult to Child role-modelling assumptions are varied.

Child to Child Social Transmission

Similar to “Adult to Adult” role-modelling, BMI outcome has little impact when Child to Child role-modelling assumptions are varied. These assumptions interact with intervention assumptions.

Infant Reported behaviours

The assumed reported infant behaviours amplify the intergenerational effects. However, these impacts are minimal.

• The proportion of mother that breastfeed impact older age groups. This is because of the increase in energy expenditure caused by breastfeeding.
• The proportion of infants that consume non-core foods and TV for more than 1 hr/day impacts younger age groups. Higher non-core food consumption and greater TV viewing increase the prevalence of overweight and obesity.

Assumed intergenerational relationships

Mortality ratios

Hazard ratios are applied to exogenous mortality rate so predicted population dynamics are maintained.

• Changing these assumptions impact older age group, primarily cause by higher mortality rates in these age groups.
• Since the primary outcome is a percentage, these ratio have little impact.

METs

Duration and intensive (METs) of physical activity are used to estimate total energy expenditure. The assumed METs for each movement category are applied to all of the population, which results in highlight sensitivity input assumptions.

• Impact from variations in METs assumptions accumulate over the population’s life course, resulting in higher impacts for older age groups.
• Increase in MET implies that each movement activity has higher intensity leading to a healthier population.
  • 1 percentage point increase in sleep METs results in a between -0.15% to -2.83% change in obesity.
  • 1 percentage point increase in inactive METs results in a between -0.14% to -2.81% change in obesity.
  • 1 percentage point increase in screen time METs results in a between -0.11% to -2.36% change in obesity.
  • 1 percentage point increase in light physical activity METs results in a between -0.08% to -1.63% change in obesity.
  • 1 percentage point increase in moderate physical activity METs results in a between -0.08% to -0.99% change in obesity.
  • 1 percentage point increase in vigorous physical activity METs results in a between 1.88% to 29.19% change in obesity.

Explanation of summary results

Light physical activity

Each behaviour is structured so that they are age dependent; observed surveyed behaviours were modelled using linear regression with age groups as the independent variable. This creates two variables for each behaviour; the intercept is the level of behaviour at the youngest age group (2 years old) and the change of behaviours over the life course.

• Changes in the intercept assumptions have cumulative impacts over the life course, resulting in higher impacts in older age groups compared to younger age groups.
• Varying age-on-age (AGE SLOPE) changes the trajectory of behaviours over the life course, making older age group BMI outcomes sensitive to changes in age slope.

Moderate physical activity

Vigorous physical activity

Screen time

Sleep

Fruit Reported Intake

Vegetables Reported Intake

Grains Reported Intake

Dairy Reported Intake

Meat and Protein Reported

Discretionary foods Reported

Fats and Oils Reported

Sugar based beverage Reported

Water Reported

Initial fat-mass (%)

Change in Male fat-mass % assumptions

Change in Female fat-mass % assumptions

Proportion of nutrients within food group Inputs

Grains

Vegetables

Fruit

Dairy

Meat and Protein

Fats

Discretionary foods

Sugar-sweetened beverages

Miscellaneous (Other)

Non-sugar-sweetened beverages

Initial BMI Prevalence Inputs

Change in Male Initial BMI Prevalence

Change in Female Initial BMI Prevalence

Explaination

Example plot for analysis outcome

Males body weight 2 - 18 year olds

Female body weight including 20-24 and 40-44